AI-Driven Scholarly Peer Review via Persistent Workflow Prompting,
  Meta-Prompting, and Meta-Reasoning
Authors: name: Evgeny Markhasin
Published: 2025-05-06 09:06:18

Abstract: Critical peer review of scientific manuscripts presents a significant
challenge for Large Language Models (LLMs), partly due to data limitations and
the complexity of expert reasoning. This report introduces Persistent Workflow
Prompting (PWP), a potentially broadly applicable prompt engineering
methodology designed to bridge this gap using standard LLM chat interfaces
(zero-code, no APIs). We present a proof-of-concept PWP prompt for the critical
analysis of experimental chemistry manuscripts, featuring a hierarchical,
modular architecture (structured via Markdown) that defines detailed analysis
workflows. We develop this PWP prompt through iterative application of
meta-prompting techniques and meta-reasoning aimed at systematically codifying
expert review workflows, including tacit knowledge. Submitted once at the start
of a session, this PWP prompt equips the LLM with persistent workflows
triggered by subsequent queries, guiding modern reasoning LLMs through
systematic, multimodal evaluations. Demonstrations show the PWP-guided LLM
identifying major methodological flaws in a test case while mitigating LLM
input bias and performing complex tasks, including distinguishing claims from
evidence, integrating text/photo/figure analysis to infer parameters, executing
quantitative feasibility checks, comparing estimates against claims, and
assessing a priori plausibility. To ensure transparency and facilitate
replication, we provide full prompts, detailed demonstration analyses, and logs
of interactive chats as supplementary resources. Beyond the specific
application, this work offers insights into the meta-development process
itself, highlighting the potential of PWP, informed by detailed workflow
formalization, to enable sophisticated analysis using readily available LLMs
for complex scientific tasks.