Support for ConversationalRetrievalChain with Azure Cognitive Search as retriever and Azure Open AI as  LLM for Streaming Output
Labels: name: auto:enhancement, name: area: models

### Feature request

Hi, I am trying to use ConversationalRetrievalChain with Azure Cognitive Search as retriever with streaming capabilities enabled. The code is not providing the output in a streaming manner. I would like to know if there is any such feature which is supported using Langchain combining Azure Cognitive Search with LLM.
The code snippet I used is as below.
 # Code Snippet

def search_docs_chain_with_memory_streaming(
        search_index_name=os.getenv("AZURE_COGNITIVE_SEARCH_INDEX_NAME"),
        question_list=[],
        answer_list=[],
):
    code = detect(question)
    language_name = map_language_code_to_name(code)
    embeddings = OpenAIEmbeddings(
        deployment=oaienvs.OPENAI_EMBEDDING_DEPLOYMENT_NAME,
        model=oaienvs.OPENAI_EMBEDDING_MODEL_NAME,
        openai_api_base=os.environ["OPENAI_API_BASE"],
        openai_api_type=os.environ["OPENAI_API_TYPE"],
    )
    memory = ConversationBufferMemory(memory_key="chat_history", output_key="answer")
    acs = AzureSearch(
        azure_search_endpoint=os.getenv("AZURE_SEARCH_SERVICE_ENDPOINT"),
        azure_search_key=os.getenv("AZURE_COGNITIVE_SEARCH_API_KEY"),
        index_name=search_index_name,
        search_type="similarity",
        semantic_configuration_name="default",
        embedding_function=embeddings.embed_query,
    )
    retriever = acs.as_retriever()
    retriever.search_kwargs = {"score_threshold": 0.8}  # {'k':1}
    print("language_name-----", language_name)
    hcp_conv_template = (
            get_prompt(workflows, "retrievalchain_hcp_conv_template1", "system_prompt", "v0")
            + language_name +
            get_prompt(workflows, "retrievalchain_hcp_conv_template2", "system_prompt", "v0")
    )
    CONDENSE_QUESTION_PROMPT = get_prompt(workflows, "retrievalchain_condense_question_prompt", "system_prompt", "v0")
    prompt = PromptTemplate(
        input_variables=["question"], template=CONDENSE_QUESTION_PROMPT
    )
    SYSTEM_MSG2 = get_prompt(workflows, "retrievalchain_system_msg_template", "system_prompt", "v0")
    messages = [
        SystemMessagePromptTemplate.from_template(SYSTEM_MSG2),
        HumanMessagePromptTemplate.from_template(hcp_conv_template),
    ]
    qa_prompt = ChatPromptTemplate.from_messages(messages)
    llm = AzureChatOpenAI(
        deployment_name=oaienvs.OPENAI_CHAT_MODEL_DEPLOYMENT_NAME, temperature=0.7, max_retries=4,
        #callbacks=[streaming_cb],
        streaming=True
        #callback_manager=CallbackManager([MyCustomHandler()])
    )
    qa_chain = ConversationalRetrievalChain.from_llm(
        llm=llm,
        retriever=retriever,
        get_chat_history=lambda o: o,
        memory=memory,
        condense_question_prompt=prompt,
        return_source_documents=True,
        verbose=True,
        #callback_manager=convo_cb_manager,
        #condense_question_llm = llm_condense_ques,
        combine_docs_chain_kwargs={"prompt": qa_prompt},
    )
    if len(question_list) == 0:
        question = question + ". Give the answer only in " + language_name + "."
    for i in range(len(question_list)):
        qa_chain.memory.save_context(
            inputs={"question": question_list[i]}, outputs={"answer": answer_list[i]}
        )
    #return qa_chain.stream({"question": question, "chat_history": []})
    return qa_chain

Also I have tried different callback handlers and invoke methods as mentioned in https://gist.github.com/jvelezmagic/03ddf4c452d011aae36b2a0f73d72f68 

Kindly suggest if there is any workaround to it. 

### Motivation

The motivation is to stream the LLM response using Langchain and Azure Cognitive Search for RAG usecase.

### Your contribution

I have attached the code and the support links in the description.