based on this paper context can you create a prompt framework template to improve llm reasoning capabilies? Constrained Reasoning Chains for Enhancing
Theory-of-Mind in Large Language Models

Abstract. Theory-of-Mind (ToM) ability possessed by Large Language
Models (LLMs) has been shown to be limited. Most existing methods
for improving ToM in LLMs adopt zero-shot prompting, and they face
challenges including poor performance in complex ToM reasoning tasks
and an inability to handle non-narrative contexts. We propose a zero-
shot prompting method named Constrained Chain-of-ToM (CCoToM)
that leverages domain knowledge and the causal relations between ToM
dimensions to address these limitations. Specifically, CCoToM guides
LLMs to construct explicit reasoning chains by first prompting LLMs to
infer related ToM dimensions (e.g., belief). Afterward, CCoToM prompts
LLMs to infer the queried ToM dimension based on the generated related
ToM dimensions and corresponding causal relations. Additionally, CCo-
ToM adaptively imposes constraints on prompts to introduce inductive
biases and improve consistency between ToM dimensions. Besides narra-
tives, CCoToM can also handle non-narrative contexts like conversations.
Extensive experiments show that CCoToM consistently outperforms pre-
vious state-of-the-art methods by large margins across all LLMs and
datasets used. We also conduct in-depth analyses to gain deeper insights
into CCoToM. We have made our code publicly available.3
Keywords: Theory-of-Mind · Large Language Models · Reasoning Chains.
1 Introduction
Theory of Mind (ToM), introduced as the ability to infer the mental states
(i.e., desires, beliefs, and intentions) of others , is crucial for human
cognition and social reasoning. Recent advancement of Large Language Models
(LLM) attracted abundant research to study the ToM in LLMs [21,26].
An example of testing the ToM in LLMs is shown in Figure 1, where an LLM
is required to infer the belief of an agent (i.e., Hiro in this example). Despite
being simple for humans, many existing ToM datasets have posed tremendous
challenges for LLMs.
Inspired by the Belief-Desire-Intention (BDI) model ], we propose a zero-
shot prompting method named Constrained Chain-of-ToM (CCoToM), which
constructs explicit reasoning chains based on ToM-specific domain knowledge
and the causal relations between ToM dimensions to address the aforementioned
limitations. Specifically, given a context and a question querying a particular
ToM dimension, such as the action of an agent, CCoToM first prompts the
underlying LLM to infer the agent’s related ToM dimensions like percept, belief,
and desire. Afterward, CCoToM prompts the LLM to infer the queried ToM
dimension based on the generated related ToM dimensions and corresponding
causal relations. In this way, CCoToM guides the LLM to construct explicit
reasoning chains throughout the ToM reasoning process, decomposing a complex
ToM reasoning task into smaller and simpler sub-tasks. Besides, motivated by
the idea of Constrained Modeling [, CCoToM adaptively imposes constraints
on prompts to introduce inductive biases and improve consistency between ToM 
Constrained Reasoning Chains for Enhancing ToM in LLMs 5
3.2 Chain-of-ToM
Every time CCoToM prompts the LLM to infer a certain ToM dimension, it only
considers the response with the highest likelihood.5 An example prompt
template of CCoToM is shown in Figure 4. CCoToM identifies the agent’s name
by prompting the LLM. For example, given a question querying an agent’s be-
lief, CCoToM prompts the LLM to answer the following question: “Whose belief
is queried in the given question?”. The other prompt templates are shown in
Section A. Now we thoroughly describe how CCoToM guides LLMs to construct
reasoning chains for different types of ToM reasoning tasks. We adopt the fol-
lowing notations in all equations: P e for percept, B for belief, D for desire, A for
action, M for the underlying LLM, and C for the given context. Additionally,
although the combinations of imposed constraints vary from prompt to prompt,
we still use a single notation S for all combinations of imposed constraints to
maintain conciseness