reviews: 
Comments: The paper introduces MaestroMotif, an extension of the recent Motif framework that accepts datasets of unlabeled demonstrations to first learn a set of labeled options, then query an LLM to write Python programs that trigger these learned options to satisfy an arbitrary objective in the given domain. MaestroMotif is compared to Motif, ordinary PPO, and expert behavior-cloning to see how well each method can satisfy arbitrary unseen objectives, and the results indicate that MaestroMotif is better than these baselines at solving long-horizon, complex requests (e.g., "go to area X, obtain item Y, wait until condition Z is met and then take action A").

The authors also present an ablation of MaestroMotif for different approaches to learning options/skills for the framework, showing the importance of learning their skills jointly and without separate policy heads. The authors also look at the scaling performance of MaestroMotif, showing that smaller LLMs produce predictably smaller returns on the complex tasks.

I thank the authors for their thorough rebuttal and for their comments on the work. After reviewing the rebuttal, I have updated my score to 8 and advocate for acceptance of the work.
Strengths: * MaestroMotif fits in well with the presented prior work and literature, and is a logical extension of much of the "code-as-policies" literature from LLMs, combining human demonstrations and data with AI planning and code-synthesis.
* The baselines used (LLM as policy, PPO, etc.) are sensible baselines and suitable cover the different methods that one might employ to attempt to learn arbitrary NetHack policies for zero-shot transfer to new skills or tasks.
* The paper presents an approach that combines large labeled datasets, human expertise, and LLM code synthesis/planning into one framework, neatly exemplifying human-AI collaboration for task completion with LLMs.
* The full details for reproducibility are in the appendix (importance of unit tests, specific prompts, re-prompting strategies, etc.), which is a useful resource for future work.
Weaknesses: * Learning the options networks seems to require intense manual effort, as a human must label different states with preferences/task-alignment to learn different options.
* While MaestroMotif outperforms the baselines for arbitrary tasks, it does seem to be pitted against methods that have no real way of generalizing to such tasks (for example, are the score-maximization approaches capable of receiving sub-goals or goals in any way?). The success of MaestroMotif is noteworthy, but the delta over prior work is less impressive given the prior work is, at times, apparently severely hamstrung.
* The overall framework relies on significant prompting/re-prompting of a 405b model, which could quickly become prohibitively expensive. It would be nice to see results or metrics on the cost (in time, memory, or money) of running MaestroMotif for each of the specified tasks, or for generalization to a new task or domain.
* Much of this paper is tailored to NetHack, so the line between "core requirement/contribution of MaestroMotif" and "tweak made to adapt to NetHack" is blurry.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3