Langchain MRKL Agent not giving useful Final Answer
Labels: name: auto:question

### Discussed in https://github.com/hwchase17/langchain/discussions/7423

<div type='discussions-op-text'>

<sup>Originally posted by **aju22** July  9, 2023</sup>
Here is the code I'm using for initializing a Zero Shot ReAct Agent with some tools for fetching relevant documents from a vector database:

```
chat_model = ChatOpenAI(
            model_name="gpt-3.5-turbo",
            temperature="0",
            openai_api_key=openai_api_key,
            streaming=True,
            # verbose=True)

llm_chain = LLMChain(llm=chat_model, prompt=prompt)

agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True, handle_parsing_errors=True)
agent_chain = AgentExecutor.from_agent_and_tools(
    agent=agent, tools=tools, verbose=True, memory=memory
)
```

However when I query for a response. 

```
query = "Can you explain a use case example of chain of thought prompting in detail?"
res = agent_chain(query)
```

This is the response I get back:

```
> Entering new  chain...
Thought: The question is asking for a detailed explanation of a use example of chain-of-thought prompting.
Action: Lookup from database
Action Input: "use example of chain-of-thought prompting"
Observation: Sure! Here's an example of chain-of-thought prompting:

Let's say we have a language model that is trained to solve math word problems. We want to use chain-of-thought prompting to improve its reasoning abilities.

The prompt consists of triples: input, chain of thought, output. For example:

Input: "John has 5 apples."
Chain of Thought: "If John gives 2 apples to Mary, how many apples does John have left?"
Output: "John has 3 apples left."

In this example, the chain of thought is a series of intermediate reasoning steps that lead to the final output. It helps the language model understand the problem and perform the necessary calculations.

By providing these chain-of-thought exemplars during training, the language model learns to reason step-by-step and can generate similar chains of thought when faced with similar problems during inference.

This approach of chain-of-thought prompting has been shown to improve the performance of language models on various reasoning tasks, including arithmetic, commonsense, and symbolic reasoning. It allows the models to decompose complex problems into manageable steps and allocate additional computation when needed.

Overall, chain-of-thought prompting enhances the reasoning abilities of large language models and helps them achieve state-of-the-art performance on challenging tasks.
Thought:I have provided a detailed explanation and example of chain-of-thought prompting.
Final Answer: Chain-of-thought prompting is a method used to improve the reasoning abilities of large language models by providing demonstrations of chain-of-thought reasoning as exemplars in prompting. It involves breaking down multi-step problems into manageable intermediate steps, leading to more effective reasoning and problem-solving. An example of chain-of-thought prompting is providing a language model with a math word problem prompt consisting of an input, chain of thought, and output. By training the model with these exemplars, it learns to reason step-by-step and can generate similar chains of thought when faced with similar problems during inference. This approach has been shown to enhance the performance of language models on various reasoning tasks.

> Finished chain.
```

As you can observe, The model has a very thorough and exact answer in it's observation. However in the next thought, the model thinks it is done providing a detailed explanation and example to the human. So the final answer is just some basic information, not really answering the question in necessary detail.

I feel like somewhere in the intermediate steps, the agent thinks it has already answered to the human, and hence just does not bother to give that as the final answer.

Can someone please help me figure out, how can I make the model output it's observation as the final answer. Or to stop making the model assume it has already answered the question to the human.

Will playing around with the prompt template work? </div>