Message History not working with LLMSingleActionAgent
Labels: name: area: memory, name: area: agent, name: auto:bug

### System Info

My systeminfo: 

langchain 0.0.294
Python 3.11

### Who can help?

Im not sure, maybe @hwchase17 and @agola11 can help

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction



That is my code: 

```python
    openai.api_key = config.openai_api_key

    llm = ChatOpenAI(temperature=1, model="gpt-3.5-turbo-16k", max_tokens=1024)

    used_tools = []

    for tool in tools_to_use:
        vector_store = get_vector_store(tool.get("collection"))
        index = VectorStoreIndex.from_vector_store(vector_store=vector_store, service_context=service_context)

        vector_store_info = VectorStoreInfo(
            content_info=tool.get("description"),
            metadata_info=[]
        )
        retriever = VectorIndexAutoRetriever(index, vector_store_info=vector_store_info)

        used_tools.append(
            Tool(
                name=tool.get("name"),
                description=tool.get("description"),
                func=lambda q: str(retriever.retrieve(q)),
            )
        )

    message_history = PostgresChatMessageHistory(
        connection_string=f"postgresql://postgres:{postgres_str}", session_id=session_id, table_name="chat_history",
    )

    prompt = CustomPromptTemplate(template=get_prompt_template(), tools=used_tools,
                                  input_variables=["input", "intermediate_steps", "chat_history"])

    output_parser = CustomOutputParser()

    memory = ConversationBufferMemory(
        memory_key="chat_history", chat_memory=message_history
    )


    llm_chain = LLMChain(llm=llm, prompt=prompt)

    tool_names = [tool.name for tool in used_tools]

    agent = LLMSingleActionAgent(
        llm_chain=llm_chain,
        output_parser=output_parser,
        stop=["\nFinal Answer:"],  # Observation
        allowed_tools=tool_names,
        handle_parsing_errors=True,
    )

    agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=used_tools, verbose=debug,
                                                        memory=memory, return_intermediate_steps=True)

    response = agent_executor(user_prompt)
``` 

### Expected behavior

If i print the params, it shows my chat history (so it's successfully loaded into code and into the agent).

```
{'intermediate_steps': [], 'stop': ['\nFinal Answer:'], 'input': 'What was the last thing I said?', 'chat_history': 'Human: ....?\nAI:....']
``` 

It should also process the chat history correctly.
But I always get this answer to my prompt:
```
Since the previous chat history is not available, I can't tell what you said last. Please give me more information or ask your question again so I can help you further.
```