Adding memory with customprompt and Custom agent is resulting in error
Labels: name: area: memory, name: area: agent, name: auto:bug

I have made a Custom Prompt template using:
```
`class CustomPromptTemplate(BaseChatPromptTemplate):
    # The template to use
    template: str
    # The list of tools available
    tools: List[Tool]
    def format_messages(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in self.tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
        kwargs["df_head"]=df.head()
        formatted = self.template.format(**kwargs)
        return [HumanMessage(content=formatted)]`
```

Calling this:
```
prompt = CustomPromptTemplate(
    template=template,
    tools=tools,
    input_variables=["input","intermediate_steps","df_head","history"]
)
```
I have also created custom agent:
```
llm_chain = LLMChain(llm=llm, prompt=prompt)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
```
Executing agent:
```
from langchain.memory import ConversationBufferWindowMemory
memory=ConversationBufferWindowMemory(memory_key='chat_history', return_messages=True, output_key='answer',k=2)
agent_executor_new = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True,memory=memory)
```
Running this with (**As per documentation**):
```
agent_executor_new.run({'input':"Generate top 3 insights on sales in Q4 and compare it against other quarters?"})
```
Results in this error:
```
`File ~/anaconda3/lib/python3.10/site-packages/langchain/chains/base.py:443, in Chain.prep_inputs(self, inputs)
    441     external_context = self.memory.load_memory_variables(inputs)
    442     inputs = dict(inputs, **external_context)
--> 443 self._validate_inputs(inputs)
    444 return inputs

File ~/anaconda3/lib/python3.10/site-packages/langchain/chains/base.py:195, in Chain._validate_inputs(self, inputs)
    193 missing_keys = set(self.input_keys).difference(inputs)
    194 if missing_keys:
--> 195     raise ValueError(f"Missing some input keys: {missing_keys}")

ValueError: Missing some input keys: {'df_head', 'history'}`

```
When I supply the missing input values:
```
agent_executor_new.run({'input':"Generate top 3 insights on sales in Q4 and compare it against other quarters?",'df_head':df.head(),'history':memory})`
```
It results in this error:

```

File ~/anaconda3/lib/python3.10/site-packages/langchain/memory/chat_memory.py:35, in BaseChatMemory.save_context(self, inputs, outputs)
>      33 def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
>      34     """Save context from this conversation to buffer."""
> ---> 35     input_str, output_str = self._get_input_output(inputs, outputs)
>      36     self.chat_memory.add_user_message(input_str)
>      37     self.chat_memory.add_ai_message(output_str)
> 
> File ~/anaconda3/lib/python3.10/site-packages/langchain/memory/chat_memory.py:22, in BaseChatMemory._get_input_output(self, inputs, outputs)
>      18 def _get_input_output(
>      19     self, inputs: Dict[str, Any], outputs: Dict[str, str]
>      20 ) -> Tuple[str, str]:
>      21     if self.input_key is None:
> ---> 22         prompt_input_key = get_prompt_input_key(inputs, self.memory_variables)
>      23     else:
>      24         prompt_input_key = self.input_key
> 
> File ~/anaconda3/lib/python3.10/site-packages/langchain/memory/utils.py:19, in get_prompt_input_key(inputs, memory_variables)
>      17 prompt_input_keys = list(set(inputs).difference(memory_variables + ["stop"]))
>      18 if len(prompt_input_keys) != 1:
> ---> 19     raise ValueError(f"One input key expected got {prompt_input_keys}")
>      20 return prompt_input_keys[0]
> ValueError: One input key expected got ['df_head', 'history', 'input']`


```
Please help!!