No support for ConversationalRetrievalChain with passing in memory and system message
Labels: name: auto:question, name: auto:improvement, name: stale, name: area: models

### System Info

The latest langchain version

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


I have written a script that initializes a chat which has access to a faiss index. I want to pass in a system prompt to the conversational agent. How do I add a system prompt to the conversational agent while making sure that the chat history is passed in and not stored and updated in memory. The code below shows details
```

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chat_models import AzureChatOpenAI


from langchain.chains import ConversationalRetrievalChain
from langchain.vectorstores import FAISS

from dotenv import load_dotenv
from langchain.schema import SystemMessage
load_dotenv()
import os

system_message = SystemMessage(
          content="You are an AI that always writes text backwards e.g. 'hello' becomes 'olleh'."
      )

embeddings = OpenAIEmbeddings(deployment="embeddings",
                              temperature=0,
                              model="text-embedding-ada-002",
                              openai_api_type="azure",
                                openai_api_version="2023-03-15-preview", 
                                openai_api_base="https://azcore-kusto-bot-openai.openai.azure.com/", 
                                openai_api_key=os.getenv("OPENAI_API_KEY"), chunk_size = 1)

vectorstore = FAISS.load_local("faiss_index", embeddings)

prompt = "Your name is AzCoreKustoCopilot"
llm=AzureChatOpenAI(deployment_name="16k-gpt-35",
                              model="gpt-35-turbo-16k",
                              openai_api_type="azure",
                                openai_api_version="2023-03-15-preview", 
                                openai_api_base="https://azcore-kusto-bot-openai.openai.azure.com/", 
                                openai_api_key=os.getenv("OPENAI_API_KEY"))

retriever = vectorstore.as_retriever()

def initialize_chat():
    # retriever = vectorstore.as_retriever()
    chat = ConversationalRetrievalChain.from_llm(llm, retriever=retriever)
    print('this is chat', chat)
    return chat

def answer_query(chat, user_query, chat_history):
    """
    user_query is the question
    chat_history is a list of lists: [[prev_prev_query, prev_prev_response], [prev_query, prev_response]]
    we convert this into a list of tuples
    """
    chat_history_tups = [tuple(el) for el in chat_history]
    print(f"user_query:{user_query}, chat_history: {chat_history_tups}")
    result = chat({"question":user_query, "chat_history": chat_history_tups})
    return result["answer"]


```


### Expected behavior

There is no way to pass in a system message and chat history to a ConversationalRetrievalChain. Why is that?