Knowledge Augmented Complex Problem Solving with Large Language Models:
  A Survey
Authors: name: Da Zheng, name: Lun Du, name: Junwei Su, name: Yuchen Tian, name: Yuqi Zhu, name: Jintian Zhang, name: Lanning Wei, name: Ningyu Zhang, name: Huajun Chen

Problem-solving has been a fundamental driver of human progress in numerous
domains. With advancements in artificial intelligence, Large Language Models
(LLMs) have emerged as powerful tools capable of tackling complex problems
across diverse domains. Unlike traditional computational systems, LLMs combine
raw computational power with an approximation of human reasoning, allowing them
to generate solutions, make inferences, and even leverage external
computational tools. However, applying LLMs to real-world problem-solving
presents significant challenges, including multi-step reasoning, domain
knowledge integration, and result verification. This survey explores the
capabilities and limitations of LLMs in complex problem-solving, examining
techniques including Chain-of-Thought (CoT) reasoning, knowledge augmentation,
and various LLM-based and tool-based verification techniques. Additionally, we
highlight domain-specific challenges in various domains, such as software
engineering, mathematical reasoning and proving, data analysis and modeling,
and scientific research. The paper further discusses the fundamental
limitations of the current LLM solutions and the future directions of LLM-based
complex problems solving from the perspective of multi-step reasoning, domain
knowledge integration and result verification.