ConversationalRetrievalChain with streaming=True => responds by repeating words and phrases
Labels: name: auto:bug

### System Info

Python: 3.10.11
LangChain: 0.0.229
SO: Windows, Linux Ubuntu and Mac

Hi people,

I'm using **ConversationalRetrievalChain** without any modifications, and in 90% of the cases, it responds by repeating words and entire phrases, like in the examples below:

Questions about Duna book:

**How did Muad'Dib die?**

Muad'Dib died of old age. **No, Muad'Dib did not die of old age**. The passage mentions that the death of Muad'Dib will be followed by the moral suicide of all men, suggesting that his death was not natural. However, the exact cause of his death is not specified in the text. No, Muad'Dib did not die of old age. The passage mentions that the death of Muad'Dib will be followed by the moral suicide of all men, suggesting that his death was not natural. **However, the exact cause of his death is not specified in the text**.

**Was Paul Atreides powerful?**

**Was Paul Atreides powerful? Yes, Paul Atreides was powerful**. He possessed oracle-like abilities and special powers that placed him beyond the usual limits of time and space. Additionally, he was a charismatic leader and skilled strategist, capable of commanding armies and achieving victories. His strength and power were recognized by the emperor and other characters in the book. Yes, Paul Atreides was powerful. **He possessed oracle-like abilities and special powers that placed him beyond the usual limits of time and space. Additionally, he was a charismatic leader and skilled strategist, capable of commanding armies and achieving victories. His strength and power were recognized by the emperor and other characters in the book.**

**PLEASE, Does anyone know about this issue?**

I have repeated several tutorials that I found on the internet, and this always happens. But strangely, I don't see anyone complaining about it in the forums, so I must be doing something wrong.

**This is the code I am using:**

llm = ChatOpenAI(model_name=app_params.Model,
                            temperature=0, 
                            openai_api_key=app_keys.OPENAI_API_KEY,
                            **streaming=True,** 
                            verbose=True,
                            callbacks=[WebSocketCallbackHandler(websocket)]         
                            ) 

memory = ConversationBufferWindowMemory(memory_key='chat_history', output_key='answer', chat_memory=chat_history, return_messages=True, k=3, verbose=True)

qna = ConversationalRetrievalChain.from_llm(
                                            llm=llm, 
                                            chain_type="stuff", 
                                            retriever=retriever, 
                                            verbose=True,
                                            memory=memory,
                                            )
#async
result = **await qna.acall**({"question": query})

**I would be very glad for any help!
Best regards,
Marcos.**

Hi, @hwchase17, I appreciate your help.

### Who can help?

Hi, @hwchase17, I appreciate your help.

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

**This is the code I am using:**

llm = ChatOpenAI(model_name=app_params.Model,
                            temperature=0, 
                            openai_api_key=app_keys.OPENAI_API_KEY,
                            **streaming=True,** 
                            verbose=True,
                            callbacks=[WebSocketCallbackHandler(websocket)]         
                            ) 

memory = ConversationBufferWindowMemory(memory_key='chat_history', output_key='answer', chat_memory=chat_history, return_messages=True, k=3, verbose=True)

qna = ConversationalRetrievalChain.from_llm(
                                            llm=llm, 
                                            chain_type="stuff", 
                                            retriever=retriever, 
                                            verbose=True,
                                            memory=memory,
                                            )
**#async**
result = **await qna.acall**({"question": query})

### Expected behavior

I hope the answer provided by ConversationalRetrievalChain makes sense and does not contain repetitions of the question or entire phrases.