Logic: Long-form Outline Generation via Imitative and Critical Self-refinement (2025)

Abstract: Long-form outline generation for expository articles requires both comprehensive knowledge coverage and logical coherence, which is essential for creating detailed Wikipedia-like content. However, existing methods face critical limitations: outlines generated in the pre-writing stage often have low knowledge density and lack detail, while retrieval-augmented approaches struggle to maintain logical coherence across retrieved information. Additionally, unlike human writers who can iteratively improve through peer feedback and reference similar topics, current approaches lack effective mechanisms for systematic outline refinement. To address these challenges, we propose Logic, a Long-form Outline Generation system via Imitative and Critical self-refinement that mimics human writers’ refinement process. Logic establishes a coherent planning framework and structured knowledge base, learns from similar topic outlines through imitation, and continuously improves through model-based critique. Experiments on FreshWiki and our dataset WikiOutline show that, compared to the best baseline, Logic’s long-form outlines are more organized (with increases of 22.85% and 21.65% respectively) and more logically coherent (with increases of 16.19% and 12.24% respectively). Human evaluation further validates Logic’s effectiveness in generating comprehensive and well-structured long-form outlines.

## introduction large language models (llms) have demonstrated remarkable capabilities in novel generation through planning-then-writing approaches @xcite @xcite . such approaches first create a coherent plot outline, then decompose the generation into subtasks, enabling llms to produce richer and more engaging content † corresponding author. figure 1 : comparison of the quality of outlines generated by different methods. the red boxes represent nodes that are too detailed and lack coherence. the outlines generated by logic achieve better results in all metrics, and the level of detail of the outline of a section is equivalent to the outline of an article by other methods. @xcite @xcite . when generating expository articles similar to wikipedia entries @xcite or reports @xcite , planning in the prewriting stage becomes even more crucial @xcite . compared with the methods that bypass the pre-writing stage, the role-playing-based method storm @xcite has enhanced the quality of the generated articles by generating outlines in the pre-writing stage. however, maintaining the coherence and knowledge richness of the outlines still remains a significant challenge. this challenge stems from three critical limitations: 1) knowledge density gap: outlines generated in the pre-writing stage often lack sufficient knowledge density and detail. 2) retrieval-coherence dilemma: while retrieval-augmented generation (rag) @xcite can introduce external knowledge to reduce the hallucination of llms @xcite , it struggles to maintain logical coherence across retrieved information. 3) limited refinement capability: unlike human writers who can iteratively improve their outlines through peer feedback @xcite and reference similar topics, current llm approaches lack effective mechanisms for systematic outline refinement @xcite and correcting factual contradictions @xcite . current approaches attempt to address these challenges through different strategies. direct generation (figure 1 (a) ) relies solely on llms' internal knowledge but produces shallow content. basic rag approaches (figure 1 (b , c )) enhance knowledge coverage but fail to improve coherence and topic coverage significantly. even advanced methods like storm @xcite , which employs multi-perspective information collection (figure 1 (d )), and omnithink @xcite , which expands knowledge boundaries through continuous reflection and exploration, still struggle with maintaining appropriate detail levels and logical consistency across the outline. inspired by human writers' practices of studying similar topics and iterative refinement, we propose logic, a novel long-form outline writing system. logic operates through three innovative mechanisms: 1) establishing structural coherence through a planning list and knowledge framework, 2) learning from similar topic outlines through imitation, 3) implementing iterative refinement based on critical model feedback. this approach enables logic to generate outlines with enhanced content richness, entity density, logical coherence, and engagement. we evaluate logic on both the freshwiki dataset @xcite and our newly proposed wikioutline dataset ( §4.1). our experiments show that the long-form outlines have significantly improved in multiple key aspects, including entity recall, interest level, coherence and organization, relevance and focus, broad coverage, logicality, breadth, and depth. human evaluation further validates our method's effectiveness in addressing the long-form outline generation challenge. our main contributions include: ## human evaluation to better understand the strengths and weaknesses of logic, we collaborated with five volunteers interested in long-form outline generation for a human evaluation. we randomly sample 10 topics from the freshwiki dataset and evaluate the long-form outlines generated by our approach and storm. to comprehensively evaluate the quality of each long-form outline, we consider four key metrics: logicality, coherence, knowledgeability, and novelty, ensuring the outlines have clear logical viewpoints, a coherent structure, sufficient knowledge of content, and novel perspectives. we calculated the intraclass correlation coefficient, and the value of icc3k was 0.73, indicating a high degree of consistency among different participants. the p-value was 0.04, suggesting that the differences among different raters were statistically significant. for the improvement of our method compared to storm in terms of the "logicality" criteria, the pearson correlation coefficient between the results of the human evaluation and the llm evaluation was calculated as 0.71. this indicates that the evaluation results of the llm and the human evaluation results are largely consistent, and there is a relatively obvious linear variation pattern between the two. figure 4 in appendix e shows the results of the human evaluation. the results indicate that logic outperforms storm across various dimensions. detailed human evaluation results are provided in appendix e. since the sophistication of content generated by llms reaches new heights, it is increasingly difficult for humans to discern the subtle differences between high-quality texts crafted by these models, human evaluation is not as obvious as automatic evaluation. however, from another perspective, it also proves the long-form outlines generated by our method are of higher quality, that is, more logical, coherent, novel, and knowledge-density-rich. ## conclusion we propose logic, a llm-based long-form outline writing framework that emulates the reflection and imitation abilities of human writers. we curate the wikioutline dataset and establish new evaluation criteria to more comprehensively study the generation of long-form outlines. both automated and human evaluations indicate that logic can generate high-quality long-form outlines that are coherent and logical. moreover, our approach is model-agnostic and can achieve good results even on an 8-billion-parameter small-scale model. this flexibility and efficiency open up new possibilities for its application in resource-constrained environments. in the future, we will further expand the application of logic in various fields and promote the automatic long-form outline writ-ing technology to develop from theoretical research to wide-ranging practical applications. and, we plan to explore fine-tuning the model with more diverse and carefully curated datasets, so as to generate more interesting, user-friendly, and objective long-form outlines. additionally, to narrow the performance gap of small-parameter models, we will investigate the application of reinforcement learning approaches to iteratively optimize small models, which will enable them to deliver highquality long-form outlines while maintaining the advantages of low computational cost and fast inference speed.